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FINDING  PARITY  IN  A  BROADCAST  NETWORK 


by  Robert  6.  Gal  lager 


I)  INTRODUCTION 

Consider  a  broadcast  network  of  N-»  1  nodes  in  which  each  binary  digit 
transmitted  by  each  node  is  received  by  each  other  node  via  a  binary 
symmetric  channei  whose  crossover  probability  c  <  1/2  is  independent  over 
transmitters,  receivers,  and  time.  Each  node  has  a  binary  state,  and  the 
problem  under  consideration  is  for  one  special  node,  called  the  receiver,  to 
determine  the  parity  of  the  set  of  node  states.  In  particular,  we  want  to 
minimize  the  number  of  binary  digits  that  must  be  sent  by  each  node  in 
order  for  the  receiver  to  determine  parity  within  some  allowable  error 
probability,  P.  We  assume  that  whatever  algorithm  is  used  for  transmitting 
the  required  information,  ail  nodes  know  the  algorithm  and  there  is  no 
contention  between  transmissions:  thus  each  binary  digit  transmitted  is 
received  and  identified  by  ail  other  nodes,  subject  to  the  noise  introduced 
by  the  binary  symmetric  channels. 

The  above  problem  was  first  formulated  by  A.  El  Gamal  [1],  and  is  of 
interest  because  it  is  one  of  the  simplest  distributed  algorithm  problems 
involving  noise.  A  closely  related  problem  that  we  treat  is  for  the  receiver 
to  find  the  state  of  each  other  node.  Note  that  the  conventional  ideas  of 
information  theory  cannot  be  used  here  because  each  node  has  only  one  bit 
of  information  to  communicate.  This  situation  of  communicating  a  limited 
amount  of  information  is  common  in  network  protocols  and  more  generally 
in  the  control  of  distributed  systems.  The  particular  character  of  the 
problem  here  comes  from  the  independence  of  the  noise  at  each  receiver  for 
a  given  transmission.  This  means  that  when  a  node  transmits  a  single 
digit,  the  other  nodes  collectively  could  make  a  good  decision  on  that  digit 
since  they  have  N  independent  noise  samples  of  it;  unfortunately,  the  nodes 
cannot  act  collectively  without  using  up  their  own  valuable  transmissions, 
which  are  also  noisy. 

The  straightforward  approach  to  this  problem  is  for  each  node  to 
broadcast  its  own  state  j  times  for  some  integer  j.  The  receiver  will  make 
an  error  in  decoding  a  given  node’s  state  with  a  probability  (j  closely  upper 

bounded  [2]  by 


(1) 


Cj  i  oc'j  where  «  = 

The  probability  P  that  the  receiver  will  make  an  error  in  calculating 
the  parity  of  the  states  is  then  upper  bounded  by  N  oTi  Since  this  bound  is 
quite  tight  for  N  o(~i  small,  we  see  that  j  must  grow  as  ln(N)  in  this 
approach  for  a  constant  P. 

2)  FINDING  PARITY  WITH  Olln(ln(N))l  BITS  PER  NODE 

The  approach  we  take  here  for  more  efficient  communication  is  to 
partition  the  nodes  (other  than  the  receiver)  into  subsets  each  with 
approximately  the  same  number  of  nodes.  In  particular,  it  is  always 
possible  to  partition  N  nodes  into  subsets  of  k  or  k-1  nodes  each  for  any  k 
satisfying  (k-l)^  i  N.  Each  node  again  broadcasts  its  own  state  j  times, 
but  then  makes  a  decision  on  the  state  of  each  of  the  other  nodes  in  its 
subset  using  the  J  receptions  from  that  node.  The  node  adds  these  decisions 
modulo  2  to  estimate  parity  for  its  own  subset,  and  then  broadcasts  this 
estimated  parity  exactly  once.  The  receiver  will  then  receive  k  or  k-l 
different  estimates  for  the  parity  of  each  subset,  and,  as  we  shall  see,  this 
allows  the  receiver  to  obtain  a  highly  reliable  decision  on  a  subset’s  parity 
with  a  relatively  small  value  of  j.  Given  the  parity  of  each  subset,  the 
parity  of  the  entire  set  is  found  by  addition  modulo  2  of  the  individual 
subset  parities. 

Note  that  the  parity  estimate  that  the  receiver  obtains  from  a  given 
node  can  be  wrong  either  because  of  noise  in  the  transmission  of  the  parity 
or  because  of  decision  errors  at  the  sending  node.  In  particular,  a  given 
received  estimate  is  incorrect  if  an  odd  number  of  errors  occur,  counting 
both  the  sending  node's  transmission  of  parity  and  its  decision  on  each  of 
the  other  states  in  the  subset;  the  probability  of  this  is  upper  bounded  by 

>  =11  -(1-«j)''''(I-2«)l/2.  (2) 

This  upper  bound  is  met  with  equality  for  a  reception  from  a  node  in  a 
subset  of  size  k,  and  with  inequality  for  size  k-l.  3  also  upper  bounds  the 
probability  that  the  receiver's  internal  estimate  of  a  subset’s  parity  is 
incorrect.  Finally  the  receiver  decides  on  the  parity  of  a  subset  by  taking  a 
majority  vote  among  the  received  estimates  and  its  own  internal  estimate. 
Since  the  errors  in  these  estimates  are  independent,  the  probability  that 


half  or  more  of  the  estimates  are  erroneous  is  upper  bounded  by 


Psubset  ^  (j) 

Combining  (2)  and  (3). 

Psubset  ‘  t'  -  (I-2«j)2<''-')(I-€)2  1^/2  (A) 

The  receiver  next  adds  the  parities  of  ail  the  subsets  (of  which  there 
are  at  most  N)  and  adds  its  own  state,  all  modulo  2.  The  probability  P  that 
this  decision  on  parity  for  the  entire  set  is  incorrect  is  upper  bounded  by 
the  probability  of  an  error  on  one  or  more  of  the  subsets,  which  is  further 
upper  bounded  by  N  Pgubset 

P  i  Nil  -  (l-2o<‘i)2(*^"*^(l-2£)2ll^''2  (5) 

in  going  from  (4)  to  (5).  we  have  used  (I)  to  upper  bound  £j  by  «~i. 

The  right  hand  side  of  (5)  first  decreases  and  then  increases  with 
increasing  k.  and  the  minimizing  integer  k  is  a  complicated  function  of  c 
and  j.  It  is  sufficient  for  our  purposes,  however,  to  simply  restrict  k  to  be 
small  enough  to  satisfy 

(l-2<x'i)2(*^"')  i  1/4  (6) 

With  this  restriction, 

P  i  where  Z  =  I  -(l-2£)/4  (7) 

This  is  essentially  the  solution  we  are  looking  for;  we  choose  k  large 
enough  to  make  the  error  probability  sufficiently  small  in  (7)  and  then 
choose  j  large  enough  to  satisfy  (6).  As  N  is  varied,  we  see  that  k  must 
increase  logarithmically  with  N,  and  then  j  increases  logarithmically  with 
k.  Since  j  and  k  must  be  integers,  however,  a  little  fussing  is  required  to 
get  a  valid  bound  on  J.  We  start  by  defining  real  number  approximations,  k 
and  f.  to  k  and  j  for  a  given  number  of  nodes  N  and  a  given  requirement  P 
on  error  probability: 

k  =  12  ln(N/P)l/ln  Z'  ’ ;  f  =  nn(2)  -  ln(  I  -2’  ’  ^^)l/ln(o<)  (8) 


We  have  used  the  notation  [x]  to  mean  the  smallest  integer  greater 
than  or  equal  to  x.  We  now  show  that  with  k  and  J  chosen  according  to  (8), 
the  resulting  error  probability  will  be  at  most  P.  Note  that  the  equation  for 
7  in  (8)  can  be  rearranged  to 

(l-2«'r)2f  =  \/A  (9) 

Since  ir>  k-l  and  j.  (6)  must  be  satisfied.  This  means  that  (7) 
must  be  satisfied,  yielding 

P  i  N  2*^/2  ^  .  p  (^0) 

The  second  inequality  above  is  valid  because  k  ^  k,  and  the  equality  is 
a  rearrangement  of  the  definition  of  k.  One  final  simplification  will  now  be 
useful  in  obtaining  our  final  bound.  Assume  that  k.  as  gi)£pn  by  (8), 
satisfies  (Til.  Then  it  is  not  hard  to  verify  that  1-2"'^*^  >  l/(2C). 
Substituting  this  into  the  definition  of  7  we  obtain 

f  i  ln(4k)  /  ln(o<)  (I  I) 

The  number  of  digits  transmitted  per  node  is  m  =  jH,  which  is  at 
most  7 *2.  Substituting  this  into  (I  I)  and  using  (8)  for  IT.  we  obtain  our 
final  bound. 


m  i  [ln(ln(N/P))  ♦  Al/ln(o<)  ♦  2 

where 

(12) 

A  =  ln(8/ln(l/Z)) 

(ts) 

Recall  that  we  have  imposed  two  restrictions  on  k  in  deriving  this 
result.  First  N  >  (k-l)^.  which  is  satisfied  If  N  >  i?,  and  second  k  i  1 . 
From  the  definition  of  (t,  these  restrictions  are 

N  i  12  ln(N/P)/ln(  1  /Z)f  ;  NiPZ'^^^  ( H) 

The  second  restriction  is  always  satisfied  for  N  >  1.  but  the  second 
restriction  is  more  substantive.  Note  first  that  (H)  is  always  satisfied 
for  large  enough  N  given  any  and  £,  and  thus  (12)  shows  that 
asymptotically,  m  increases  at  most  as  ln(ln(N))  On  the  other  hand,  for 
given  N  and  €.  (M)  is  always  violated  for  small  enough  Thus  (12) 


(subject  to  (H))  does  not  show  that  m  asymptotically  varies  with  P  as 
ln(lr<1/^).  This  is  reassuring,  since  even  if  all  nodes  other  than  the 
receiver  knew  the  parity  of  the  states,  the  error  probability  could  not 
decrease  faster  than  Actually,  by  changing  the  strategy  somewhat 
and  making  all  subsets  of  size  k  except  for  one  subset  of  size  between  I 
and  k.  the  restriction  N  i  (k-l)^  could  be  relaxed  to  N  >  k.  In  this  case, 
some  of  the  nodes  would  have  to  transmit  an  extra  digit  to  help  resolve  the 
parity  of  the  small  subset,  and  the  bound  on  m  would  be  somewhat 
weakened.  We  omit  the  details  of  this  since  it  is  tedious  and  doesn't 
Improve  the  asymptotic  behavior  with  N. 

it  is  also  possible  to  reduce  the  value  of  A  for  large  values  of  e  by 
having  each  node  transmit  an  estimate  of  parity  for  several  subsets  rather 
than  just  its  own  subset.  Again  we  omit  the  analysis  since  it  is  tedious 
and  does  not  materially  improve  the  result. 

3)  FINDING  THE  STATE  OF  ALL  NODES 

In  this  section  we  show  that  the  receiver  can  determine  the  state  of 
all  nodes  with  very  few  more  transmissions  per  node  than  are  required  to 
determine  parity.  Our  strategy  in  doing  this  is  to  form  a  set  of  N  subsets 
of  the  N  nodes  (not  counting  the  receiver)  in  such  a  way  that  each  subset 
contains  k-l  or  k  nodes  for  some  k  and  each  node  is  contained  in  k-1  or  k 
subsets.  Furthermore,  we  constrain  the  choice  of  subsets  so  that  no  pair  of 
nodes  appear  together  in  more  than  one  subset.  In  the  appendix,  we  show 
that  such  a  set  of  subsets  can  always  be  constructed  if 

N  i  2k(k-1)2  (15) 

We  next  associate  each  node  with  one  subset  in  a  one  to  one  fashion 
so  that  each  node  is  associated  with  a  subset  containing  it  and  each  subset 
has  one  of  its  contained  nodes  associated  with  it.  The  appendix  also  shows 
how  this  association  can  be  constructed.  Each  node  then  sends  its  own 
state  j  times  and  then  each  node  estimates  the  parity  of  its  associated 
subset  in  the  same  way  as  before.  Finally  each  node  sends  the  parity  of  its 
associated  subset  and  the  receiver  uses  this  information,  plus  its  own 
receptions  of  the  node  states,  to  determine  the  state  of  each  node.  Thus 
each  node  sends  m  =  j-^  I  binary  digits  as  before. 

Now  consider  how  the  receiver  can  decode  the  state  of  each  node  from 
the  received  information.  First  the  receiver  makes  an  internal  decision  on 


the  state  of  each  digit  from  the  j  noisy  receptions  of  that  digit.  The 
probability  of  error  for  each  of  these  internal  decisions  is  6j  i  oc’i  as 

before.  In  order  to  make  a  final  decoding  of  the  state  of  a  given  node,  say 
node  i.  the  receiver  considers  each  of  the  subsets,  say  Sjj.  Sj  2 . or 

Sj  j . Sj  that  contain  node  i.  For  a  given  subset,  say  S{  2^,  the 

receiver  modulo  2  adds  its  internal  decisions  on  the  nodes  in  S{  -  (i)  to 

the  received  parity  estimate  of  Sj  Note  that  if  the  receivers  interna] 

decisions  on  these  nodes  are  all  correct,  and  if  the  transmitting  nodes 
decisions  on  the  nodes  In  the  subset  are  all  correct,  and  if  the  transmission 
of  that  parity  is  correct,  then  this  modulo  2  sum  is  simply  the  state  of 
node  i  since  all  other  node  states  are  added  twice.  Thus  this  sum  is  in 
some  sense  an  estimate  of  the  state  of  node  i.  More  particularly,  the 
probability  that  this  estimate  is  incorrect  is  the  probability  of  an  odd 
number  of  errors  in  the  transmission  of  parity,  in  the  receiver's  internal 
decisions  on  the  nodes  in  Sj  and  in  the  transmitting  node's  decisions  on 

the  nodes  of  the  subset  other  than  itself.  If  the  subset  contains  k  nodes, 
then  we  are  looking  at  k-l  decisions  at  the  receiver,  k-l  at  the 
transmitting  node,  and  one  transmission  of  parity.  The  probability  that  this 
estimate  is  incorrect  is  then 

fl'  =  II -(l-2£j)2(''‘’)(l-2€)l/2  (16) 

If  the  subset  contains  k-1  nodes,  then  k-l  would  be  replaced  by  k'2  in 
the  above  equation,  so  that  3’  is  an  upper  bound  on  the  probability  of  an 
incorrect  estimate  in  that  case.  Finally,  3'  also  upper  bounds  the 
probability  of  an  error  in  the  receiver's  internal  estimate  of  the  state  of 
node  i.  The  receiver  now  has  its  internal  decision  of  node  i's  state  plus 
either  k  or  k-l  estimates  from  the  different  subsets  containing  i.  Since  no 
two  subsets  contain  more  than  one  node  in  common,  Sj  j-{i}.  Sj  —  are 

all  disjoint  and  all  of  these  estimates  are  based  on  mutually  independent 
errors.  Thus  when  the  receiver  takes  a  majority  vote  on  its  k  or  k-^  I 
estimates,  the  probability  of  error  in  this  final  decision  on  node  i  is  upper 
bounded  by 

P|<  (17) 

Combining  (16)  and  (17)  and  upper  bounding  cj  with  o(~J.  we  get 


PjS  [I  -  (l-2c<‘))^^''''^(l-2t)^F^ 


(18) 


The  probability  that  any  node  state  will  be  decoded  incorrectly  is  now 
upper  bounded  by  P  ^  N  Pj.  We  now  restrict  Ic  to  be  small  enough  that 


(i-2o<-i)<''-') 

i  1/4  yielding 

(19) 

P  i 

2=1-  (l-2£)2 

(20) 

As  in  the  analysis  of  finding  the  parity  of  the  states,  we  now  consider 
a  required  error  probability.  P,  and  choose  k  and  j  to  meet  the  requirement. 

k  =  2  ln(N/^/ln(l/2);  f  =  [ln(2)  -  ln(l-2”^''(2^1/ln(<x)  (21) 

k=  r^i;  j  =  rn 

As  before,  this  guarantees  that  (19)  is  satisfied  and  that  the  error 
probability  is  at  most  P.  If  2k  >  1.  we  can  upper  bound  f  by  In(4i0/ln(o(). 
Since  the  number  of  digits  per  node  is  at  most  J*2,  we  can  use  this  bound 
on  f  with  the  definition  of  k  to  obtain 

m  i  llnCHN/P))  *  A‘l/ln(o<)  ♦  2  where  (22) 

A'  =  ln(16/ln(l/Z))  (23) 

Note  that  the  required  number  of  transmission  here  exceeds  that  for 
finding  parity  only  by  ln(2)/ln(«).  Recall  that  the  construction  here 
required  N  j  2k(k-l)^.  which  is  valid  for  N  2  2j?(k+l).  From  (21).  we  see 
that  this  is  satisfied,  for  any  given  ^  and  £  for  all  sufficiently  large  N.  As 
before,  for  fixed  N  and  £.  the  restriction  is  always  violated  for  small 
enough  P. 

An  interesting  question  raised  by  these  results  is  whether  strategies 
exist  for  which  m  can  grow  asymptotically  even  more  slowly  than  as 
ln(ln(N)).  We  conjecture  that  the  answer  is  no. 


APPENDIX 


We  start  with  a  set  of  N  nodes  and  want  to  construct  N  subsets  of  k  or 
k- 1  nodes  each  with  the  property  that  each  node  is  in  k  or  k- 1  subsets  and  no 
two  subsets  contain  any  pair  of  nodes  in  common.  Assume  that  N  i  2k(k-i  )^. 
Let  L  =  [N/kl  and  let  N'  =  Lk.  We  first  construct  N'  subsets  of  k  nodes 
each  from  a  set  of  N’  nodes  and  then  delete  N'-N  nodes  and  subsets. 

Consider  the  problem  in  terms  of  an  N'  by  N'  matrix  of  O's  and  Ts. 
The  columns  correspond  to  nodes  and  the  rows  to  subsets.  The  I's  in  a  row 
correspond  to  the  nodes  in  the  corresponding  subset,  so  our  problem  is  to 
construct  an  N'  by  N'  matrix  in  which  each  row  and  each  column  contains  k 
I's  and  for  any  two  rows,  there  is  at  most  one  column  for  which  both  rows 
contain  1  ‘s.  We  construct  such  a  matrix  by  first  considering  the  N’  by  N' 
matrix  as  partitioned  into  k^  L  by  L  permutation  matrices.  The  fact  that 
each  L  by  L  matrix  is  a  permutation  matrix  guarantees  that  each  and 
each  column  of  the  entire  matrix  contains  k  ones.  The  figure  below  shows 
such  a  partitioned  matrix  where  L  =  3  and  k  =  3;  note  that  no  two  rows 
contain  more  than  a  single  I  in  common. 


Next  consider  the  construction  of  the  L  by  L  permutation  submatrices. 
Suppose  we  have  already  selected  some  of  them  and  are  selecting  a  new 
one.  For  a  given  row,  there  are  at  most  k-l  I's  already  in  that  row  (in  the 


other  submatrices),  and  for  each  of  those  I’s,  there  are  at  most  k-1  other 
rows  that  have  1  's  in  the  same  column.  Each  of  these  (at  most  (k- 1  )^) 
rows  have  at  most  a  single  one  in  the  colums  of  the  new  permutation 
submatrix  being  constructed,  and  these  columns  are  unavailable  for  the 
given  row  of  the  permutation  matrix;  if  a  1  was  placed  in  one  of  these 
columns  in  the  given  row,  then  that  row  and  another  row  would  have  two 
1  's  in  common.  Using  the  same  argument  on  columns,  we  see  that  each 
column  of  the  submatrix  has  at  most  (k-1)^  row  positions  unavailable  for 
rs.  The  constraint  N  i  2k(k-l)2  implies  that  L  i  2(k-l)2,  and  thus  each 
row  and  column  has  at  least  L/2  positions  available  for  I’s  without 
violating  the  condition  that  no  two  rows  have  two  I's  in  commo. 

Now  suppose  we  mark  all  the  available  positions  in  the  new 
submatrix  and  start  to  construct  a  permutation  matrix  using  only  available 
positions  for  I's;  the  problem  then  is  to  place  a  single  1  in  each  row  and 
column,  using  only  available  positions.  We  place  I's  into  the  submatrix  one 
at  a  time,  and  after  each  I  is  placed,  the  corresponding  row  and  column  are 
considered  as  blocked.  Thus  at  each  step,  we  look  for  an  available  position 
for  which  the  row  and  column  are  unblocked.  If  the  procedure  terminates 
with  a  1  in  each  row  and  column,  the  permutation  submatrix  is  complete 
and  we  proceed  to  the  next  submatrix  until  all  submatrices  are  complete. 
Suppose,  however,  that  at  some  point  there  is  at  least  one  unblocked  row 
and  column  with  no  available  unblocked  positions  in  either.  This  row  and 
column  each  have  at  least  L/2  available  but  blocked  positions.  The 
submatrix  at  this  point  contains  at  most  L-1  I's.  Thus  at  least  one  of  the 
rs  in  the  submatrix  must  be  blocking  both  an  available  position  in  the  given 
row  and  an  available  position  in  the  given  column.  Thus  by  removing  this 
single  I ,  we  can  add  two  1  's,  one  in  the  given  row  and  another  in  the  given 
column.  By  repeating  this  procedure  if  need  be,  the  given  permutation 
submatrix  can  be  completely  constructed.  The  figure  below  illustrates  this 
procedure.  An  A  indicates  a  position  that  is  available  but  in  a  blocked  row 
or  column  and  a  U  indicates  an  unavailable  position. 

The  constraint  L  i  (k-l)^  that  we  have  imposed  appears  to  be  stronger 
than  necessary  in  many  cases.  For  example,  if  L  is  prime  and  L  =  k,  it  is 
possible  to  construct  a  matrix  with  the  required  properties  as  follows:  let 
Pj  denote  the  cyclic  L  by  L  permutation  matrix  that  is  shifted  i  places  from 

the  identity  matrix.  Then  for  the  m^^  row  and  n^*^  column  submatrix,  use  Pj 

where  i  =  (m-1)(n-l). 
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Figure  2 


The  above  paragraphs  show  how  to  construct  an  N’  by  N'  matrix  where 
N’  =  k  L  i  N.  By  removing  the  first  N'-N  rows  and  columns,  we  have  an  N  by 
N  matrix.  Since  N’-N  <  L,  we  have  removed  at  most  a  single  1  from  each 
row  and  column,  so  the  remaining  matrix  satisfies  the  required  properties 
and  has  either  k  or  k-1  I's  in  each  row  and  column.  Finally,  we  can  make 
the  top  left  submatrix  be  an  Identity  matrix  in  the  original  construction. 

We  then  associate  each  column  of  the  N  by  N  matrix  with  the  row  for  which 
the  main  diagonal  of  submatrices  cw^tains  a  1.  Each  row  is  clearly  chosen 
once  and  only  once  by  this  construction. 
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